The Safety INdEx of Prehospital On Scene Triage (SINEPOST) study: The development and validation of a risk prediction model to support ambulance clinical transport decisions on-scene.

One of the main problems currently facing the delivery of safe and effective emergency care is excess demand, which causes congestion at different time points in a patient's journey. The modern case-mix of prehospital patients is broad and complex, diverging from the traditional 'time critical accident and emergency' patients. It now includes many low-acuity patients and those with social care and mental health needs. In the ambulance service, transport decisions are the hardest to make and paramedics decide to take more patients to the ED than would have a clinical benefit. As such, this study asked the following research questions: In adult patients attending the ED by ambulance, can prehospital information predict an avoidable attendance? What is the simulated transportability of the model derived from the primary outcome? A linked dataset of 101,522 ambulance service and ED ambulance incidents linked to their respective ED care record from the whole of Yorkshire between 1st July 2019 and 29th February 2020 was used as the sample for this study. A machine learning method known as XGBoost was applied to the data in a novel way called Internal-External Cross Validation (IECV) to build the model. The results showed great discrimination with a C-statistic of 0.81 (95%CI 0.79-0.83) and excellent calibration with an O:E ratio was 0.995 (95% CI 0.97-1.03), with the most important variables being a patient's mobility, their physiological observations and clinical impression with psychiatric problems, allergic reactions, cardiac chest pain, head injury, non-traumatic back pain, and minor cuts and bruising being the most important. This study has successfully developed a decision-support model that can be transformed into a tool that could help paramedics make better transport decisions on scene, known as the SINEPOST model. It is accurate, and spatially validated across multiple geographies including rural, urban, and coastal. It is a fair algorithm that does not discriminate new patients based on their age, gender, ethnicity, or decile of deprivation. It can be embedded into an electronic Patient Care Record system and automatically calculate the probability that a patient will have an avoidable attendance at the ED, if they were transported. This manuscript complies with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement (Moons KGM, 2015).

The Safety INdEx of Prehospital On Scene Triage (SINEPOST) study: the development and validation of a risk prediction model to support ambulance clinical transport decisions on-scene-a protocol.

BACKGROUND: Demand for both the ambulance service and the emergency department (ED) is rising every year and when this demand is excessive in both systems, ambulance crews queue at the ED waiting to hand patients over. Some transported ambulance patients are 'low-acuity' and do not require the treatment of the ED. However, paramedics can find it challenging to identify these patients accurately. Decision support tools have been developed using expert opinion to help identify these low acuity patients but have failed to show a benefit beyond regular decision-making. Predictive algorithms may be able to build accurate models, which can be used in the field to support the decision not to take a low-acuity patient to an ED. METHODS AND ANALYSIS: All patients in Yorkshire who were transported to the ED by ambulance between July 2019 and February 2020 will be included. Ambulance electronic patient care record (ePCR) clinical data will be used as candidate predictors for the model. These will then be linked to the corresponding ED record, which holds the outcome of a 'non-urgent attendance'. The estimated sample size is 52,958, with 4767 events and an EPP of 7.48. An XGBoost algorithm will be used for model development. Initially, a model will be derived using all the data and the apparent performance will be assessed. Then internal-external validation will use non-random nested cross-validation (CV) with test sets held out for each ED (spatial validation). After all models are created, a random-effects meta-analysis will be undertaken. This will pool performance measures such as goodness of fit, discrimination and calibration. It will also generate a prediction interval and measure heterogeneity between clusters. The performance of the full model will be updated with the pooled results. DISCUSSION: Creating a risk prediction model in this area will lead to further development of a clinical decision support tool that ensures every ambulance patient can get to the right place of care, first time. If this study is successful, it could help paramedics evaluate the benefit of transporting a patient to the ED before they leave the scene. It could also reduce congestion in the urgent and emergency care system. TRIAL REGISTRATION: This study was retrospectively registered with the ISRCTN: 12121281.

Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: a systematic review.

BACKGROUND: The primary objective of this review is to assess the accuracy of machine learning methods in their application of triaging the acuity of patients presenting in the Emergency Care System (ECS). The population are patients that have contacted the ambulance service or turned up at the Emergency Department. The index test is a machine-learning algorithm that aims to stratify the acuity of incoming patients at initial triage. This is in comparison to either an existing decision support tool, clinical opinion or in the absence of these, no comparator. The outcome of this review is the calibration, discrimination and classification statistics. METHODS: Only derivation studies (with or without internal validation) were included. MEDLINE, CINAHL, PubMed and the grey literature were searched on the 14th December 2019. Risk of bias was assessed using the PROBAST tool and data was extracted using the CHARMS checklist. Discrimination (C-statistic) was a commonly reported model performance measure and therefore these statistics were represented as a range within each machine learning method. The majority of studies had poorly reported outcomes and thus a narrative synthesis of results was performed. RESULTS: There was a total of 92 models (from 25 studies) included in the review. There were two main triage outcomes: hospitalisation (56 models), and critical care need (25 models). For hospitalisation, neural networks and tree-based methods both had a median C-statistic of 0.81 (IQR 0.80-0.84, 0.79-0.82). Logistic regression had a median C-statistic of 0.80 (0.74-0.83). For critical care need, neural networks had a median C-statistic of 0.89 (0.86-0.91), tree based 0.85 (0.84-0.88), and logistic regression 0.83 (0.79-0.84). CONCLUSIONS: Machine-learning methods appear accurate in triaging undifferentiated patients entering the Emergency Care System. There was no clear benefit of using one technique over another; however, models derived by logistic regression were more transparent in reporting model performance. Future studies should adhere to reporting guidelines and use these at the protocol design stage. REGISTRATION AND FUNDING: This systematic review is registered on the International prospective register of systematic reviews (PROSPERO) and can be accessed online at the following URL: https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42020168696This study was funded by the NIHR as part of a Clinical Doctoral Research Fellowship.

Using vignettes to assess the accuracy and rationale of paramedic decisions on conveyance to the emergency department.

INTRODUCTION: Paramedics make important decisions about whether a patient needs transport to hospital, or can be discharged on scene. These decisions require a degree of accuracy, as taking low acuity patients to the emergency department (ED) can support ambulance ramping. In contrast, leaving mid-high acuity patients on scene can lead to incidents and recontact. This study aims to investigate the accuracy of conveyance decisions made by paramedics when looking at real life patient scenarios with known outcomes. It also aims to explore how the paramedic made the decision. METHODS: We undertook a prospective mixed method triangulation design. Six individual patient vignettes were created using linked ambulance and ED data. These were then presented in an online survey to paramedics in Yorkshire. Half the vignettes related to mid-high acuity attendances at the ED and the other half were low acuity. Vignettes were validated by a small expert panel. Participants were asked to determine the appropriate conveyance decision and to explain the rationale behind their decisions using a free-text box. RESULTS: A total of 143 paramedics undertook the survey and 858 vignettes were completed. There was clear agreement between paramedics for transport decisions (ƙ = 0.63). Overall accuracy was 0.69 (95% CI 0.66-0.73). Paramedics were better at 'ruling in' the ED, with sensitivity of 0.89 (95% CI 0.86-0.92). The specificity of 'ruling out' the ED was 0.51 (95% CI 0.46-0.56). Text comments were focused on patient safety and risk aversion. DISCUSSION: Paramedics make accurate conveyance decisions but are more likely to over-convey than under-convey, meaning that while decisions are safe they are not always appropriate. It is important that paramedics feel supported by the service to make safe and confident non-conveyance decisions. Reducing over-conveyance is a potential method of reducing demand in the urgent and emergency care system.

A service evaluation of paediatric pain management in an English ambulance service.

INTRODUCTION: Evidence from the past 20 years has highlighted that acute pain is not managed well in the emergency setting, in particular with children. Inadequate management of pain can result in long-term changes in both physical and mental health. This service evaluation aimed to determine how paediatric pain is assessed and managed by ambulance clinicians in a large region in England. METHODS: This retrospective service evaluation analysed electronic patient record (ePR) data routinely collected between September and December 2018. All paediatric patients (< 18 years of age) with pain documented narratively, or a pain score of ≥ 1/10, were included. The primary outcome measure was the proportion of patients with severe pain (defined as a pain score of ≥ 7/10) who achieve effective pain management (reduction in pain score of ≥ 2/10). RESULTS: A total of 2801 paediatric patients who had documented pain were included in the analysis and the median age of patients was three years (interquartile range, 1-12 years). Most had a medical cause of pain (2387/2801, 85.2%), and analgesia was administered by the ambulance crew in 403/2801 (14.4%) patients. Multiple pain scores were recorded for 667 patients. Effective pain management was achieved in 233/271 (86%) patients in moderate pain and 204/210 (97.1%) patients in severe pain. However, of the 437 children in moderate to severe pain who achieved effective pain management, 381 (87%) received no analgesia. CONCLUSION: Children in severe pain received effective pain management, despite the majority not receiving any analgesia. This should be investigated further since non-pharmacological methods of analgesia are unlikely to explain a reduction of this magnitude. Ambulance staff need to be encouraged to record a pain score promptly after arriving on scene and ensure it is repeated. Pain score should be documented as part of the physiological observations and not in the free text of ePRs to ensure that it is identified during audits.